← Back to Library

Math and Architectures of Deep Learning
Copyright
front matter
- foreword
- contents
- preface
- acknowledgments
- about this book
- about the authors
- about the cover illustration
1 An overview of machine learning and deep learning
- 1.1 A first look at machine/deep learning: A paradigm shift in computation
- 1.2 A function approximation view of machine learning:Models and their training
- 1.3 A simple machine learning model: The cat brain
- 1.4 Geometrical view of machine learning
- 1.5 Regression vs. classification in machine learning
- 1.6 Linear vs. nonlinear models
- 1.7 Higher expressive power through multiple nonlinear layers: Deep neural networks
- Summary
2 Vectors, matrices, and tensors in machine learning
- 2.1 Vectors and their role in machine learning
  - 2.1.1 The geometric view of vectors and its significance in machine learning
- 2.2 PyTorch code for vector manipulations
  - 2.2.1 PyTorch code for the introduction to vectors
- 2.3 Matrices and their role in machine learning
  - 2.3.1 Matrix representation of digital images
- 2.4 Python code: Introducing matrices, tensors, and images via PyTorch
- 2.5 Basic vector and matrix operations in machine learning
- 2.6 Orthogonality of vectors and its physical significance
- 2.7 Python code: Basic vector and matrix operations via PyTorch
- 2.8 Multidimensional line and plane equations and machine learning
  - 2.8.1 Multidimensional line equation
  - 2.8.2 Multidimensional planes and their role in machine learning
- 2.9 Linear combinations, vector spans, basis vectors, and collinearity preservation
- 2.10 Linear transforms: Geometric and algebraic interpretations
  - 2.10.1 Generic multidimensional definition of linear transforms
  - 2.10.2 All matrix-vector multiplications are linear transforms
- 2.11 Multidimensional arrays, multilinear transforms, and tensors
  - 2.11.1 Array view: Multidimensional arrays of numbers
- 2.12 Linear systems and matrix inverse
- 2.13 Eigenvalues and eigenvectors: Swiss Army knives of machine learning
- 2.14 Orthogonal (rotation) matrices and their eigenvalues and eigenvectors
- 2.15 Matrix diagonalization
- 2.16 Spectral decomposition of a symmetric matrix
  - 2.16.1 PyTorch code for the spectral decomposition of a matrix
- 2.17 An application relevant to machine learning: Finding the axes of a hyperellipse
  - 2.17.1 PyTorch code for hyperellipses
- Summary
3 Classifiers and vector calculus
- 3.1 Geometrical view of image classification
- 3.2 Error, aka loss function
- 3.3 Minimizing loss functions: Gradient vectors
  - 3.3.1 Gradients: A machine learning-centric introduction
  - 3.3.2 Level surface representation and loss minimization
- 3.4 Local approximation for the loss function
  - 3.4.1 1D Taylor series recap
  - 3.4.2 Multidimensional Taylor series and the Hessian matrix
- 3.5 PyTorch code for gradient descent, error minimization,and model training
- 3.6 Convex and nonconvex functions, and global and local minima
- 3.7 Convex sets and functions
- Summary
4 Linear algebraic tools in machine learning
- 4.1 Distribution of feature data points and true dimensionality
- 4.2 Quadratic forms and their minimization
  - 4.2.1 Minimizing quadratic forms
  - 4.2.2 Symmetric positive (semi)definite matrices
- 4.3 Spectral and Frobenius norms of a matrix
  - 4.3.1 Spectral norms
  - 4.3.2 Frobenius norms
- 4.4 Principal component analysis
- 4.5 Singular value decomposition
- 4.6 Machine learning application: Document retrieval
- Summary
5 Probability distributions in machine learning
- 5.1 Probability: The classical frequentist view
  - 5.1.1 Random variables
  - 5.1.2 Population histograms
- 5.2 Probability distributions
- 5.3 Basic concepts of probability theory
- 5.4 Joint probabilities and their distributions
  - 5.4.1 Marginal probabilities
  - 5.4.2 Dependent events and their joint probability distribution
- 5.5 Geometrical view: Sample point distributions for dependent and independent variables
- 5.6 Continuous random variables and probability density
- 5.7 Properties of distributions: Expected value, variance, and covariance
  - 5.7.1 Expected value (aka mean)
  - 5.7.2 Variance, covariance, and standard deviation
- 5.8 Sampling from a distribution
- 5.9 Some famous probability distributions
- Summary
6 Bayesian tools for machine learning
- 6.1 Conditional probability and Bayes’ theorem
- 6.2 Entropy
  - 6.2.1 Geometrical intuition for entropy
  - 6.2.2 Entropy of Gaussians
- 6.3 Cross-entropy
- 6.4 KL divergence
  - 6.4.1 KLD between Gaussians
- 6.5 Conditional entropy
  - 6.5.1 Chain rule of conditional entropy
- 6.6 Model parameter estimation
- 6.7 Latent variables and evidence maximization
- 6.8 Maximum likelihood parameter estimation for Gaussians
  - 6.8.1 Python PyTorch code for maximum likelihood estimation
  - 6.8.2 Python PyTorch code for maximum likelihood estimation using gradient descent
- 6.9 Gaussian mixture models
- Summary
7 Function approximation: How neural networks model the world
- 7.1 Neural networks: A 10,000-foot view
- 7.2 Expressing real-world problems: Target functions
- 7.3 The basic building block or neuron: The perceptron
- 7.4 Toward more expressive power: Multilayer perceptrons (MLPs)
  - 7.4.1 MLP for logical XOR
- 7.5 Layered networks of perceptrons: MLPs or neural networks
- Summary
8 Training neural networks: Forward propagation and backpropagation
- 8.1 Differentiable step-like functions
  - 8.1.1 Sigmoid function
  - 8.1.2 Tanh function
- 8.2 Why layering?
- 8.3 Linear layers
  - 8.3.1 Linear layers expressed as matrix-vector multiplication
  - 8.3.2 Forward propagation and grand output functions for an MLP of linear layers
- 8.4 Training and backpropagation
- 8.5 Training a neural network in PyTorch
- Summary
9 Loss, optimization, and regularization
- 9.1 Loss functions
- 9.2 Optimization
- 9.3 Regularization
- Summary
10 Convolutions in neural networks
- 10.1 One-dimensional convolution: Graphical and algebraical view
- 10.2 Convolution output size
- 10.3 Two-dimensional convolution: Graphical and algebraic view
- 10.4 Three-dimensional convolution
  - 10.4.1 Video motion detection via 3D convolution
  - 10.4.2 PyTorch- Three-dimensional convolution with custom weights
- 10.5 Transposed convolution or fractionally strided convolution
- 10.6 Adding convolution layers to a neural network
  - 10.6.1 PyTorch- Adding convolution layers to a neural network
- 10.7 Pooling
- Summary
11 Neural networks for image classification and object detection
- 11.1 CNNs for image classification: LeNet
  - 11.1.1 PyTorch- Implementing LeNet for image classification on MNIST
- 11.2 Toward deeper neural networks
- 11.3 Object detection: A brief history
- 11.4 Faster R-CNN: A deep dive
- Summary
12 Manifolds, homeomorphism, and neural networks
- 12.1 Manifolds
  - 12.1.1 Hausdorff property
  - 12.1.2 Second countable property
- 12.2 Homeomorphism
- 12.3 Neural networks and homeomorphism between manifolds
- Summary
13 Fully Bayes model parameter estimation
- 13.1 Fully Bayes estimation: An informal introduction
  - 13.1.1 Parameter estimation and belief injection
- 13.2 MLE for Gaussian parameter values recap)
- 13.3 Fully Bayes parameter estimation: Gaussian, unknown mean, known precision
- 13.4 Small and large volumes of training data, and strong and weak priors
- 13.5 Conjugate priors
- 13.6 Fully Bayes parameter estimation: Gaussian, unknown precision, known mean
  - 13.6.1 Estimating the precision parameter
- 13.7 Fully Bayes parameter estimation: Gaussian, unknown mean, unknown precision
  - 13.7.1 Normal-gamma distribution
  - 13.7.2 Estimating the mean and precision parameters
- 13.8 Example: Fully Bayesian inferencing
  - 13.8.1 Maximum likelihood estimation
  - 13.8.2 Bayesian inference
- 13.9 Fully Bayes parameter estimation: Multivariate Gaussian, unknown mean, known precision
- 13.10 Fully Bayes parameter estimation: Multivariate, unknown precision, known mean
  - 13.10.1 Wishart distribution
  - 13.10.2 Estimating precision
- Summary
14 Latent space and generative modeling, autoencoders, and variational autoencoders
- 14.1 Geometric view of latent spaces
- 14.2 Generative classifiers
- 14.3 Benefits and applications of latent-space modeling
- 14.4 Linear latent space manifolds and PCA
  - 14.4.1 PyTorch code for dimensionality reduction using PCA
- 14.5 Autoencoders
  - 14.5.1 Autoencoders and PCA
- 14.6 Smoothness, continuity, and regularization of latent spaces
- 14.7 Variational autoencoders
- Summary
Appendix
- A.1 Dot product and cosine of the angle between two vectors
- A.2 Determinants
- A.3 Computing the variance of a Gaussian distribution
- A.4 Two theorems in statistics
  - A.4.1 Jensen’s Inequality
  - A.4.2 Log sum inequality
- A.5 Gamma functions and distribution
  - A.5.1 Gamma function
  - A.5.2 Gamma distribution
notations
Index

Math and Architectures of Deep Learning

Krishnendu Chaudhury with Ananya H. Ashok, Sujay Narumanchi, and Devashish Shankar

Foreword by Prith Banerjee

To comment go to liveBook

Manning

Shelter Island

For more information on this and other Manning titles go to

www.manning.com

← Previous Section 2 of 22 Next →